Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature sentinel #131

Open
wants to merge 7 commits into
base: 3.x
Choose a base branch
from
Open

Feature sentinel #131

wants to merge 7 commits into from

Conversation

sartor
Copy link

@sartor sartor commented Jul 12, 2022

Redis sentinel master auto discovery
This is alpha version, it may contains major bugs. Use it only on own risk!
Introduced new class SentinelClient
with public methods:
masterAddress() - for simple master address discovery from sentinel
masterConnection() - for complete check for master connection according to https://redis.io/docs/reference/sentinel-clients/

Some tests included. I've changed ci.yml to support sentinel tests
Related to #69

@SimonFrings
Copy link
Contributor

Hey @sartor, thanks for opening up this ticket 👍

This is definitely a useful feature for this project. In most use cases you already have a Redis infrastructure when adding a sentinel, which means the sentinel feature in this project has to work properly.

If you need anything from our side, we're happy to help to get this shipped! 🎉

@sartor
Copy link
Author

sartor commented Aug 2, 2022

Something wrong with sentinel tests in github environment. Local ip address is looking like invalid IPv6. I'll check it later. May be you any have ideas?

@qlereboursBS
Copy link

Hey @sartor ! I'm trying to use the code of your PR to add the redis sentinel support for my app.
I just don't understand how it works when it comes to contacting the redis sentinels to know the master address.
I checked on internet and started to look at other libs, I'm not sure that a HTTP call could allow us to retrieve the master address.
In the CLI it would be redis-cli SENTINEL get-master-addr-by-name but I'm not sure it's accessible from an HTTP call.

Can you give more details about what you created please?
Thanks

@sartor
Copy link
Author

sartor commented Mar 24, 2024

Hi. It is not http call. My code construct some kind of dsn address (connection uri with parameters) before connection to redis sentinels. After connection retrieved it calls for some commands according to https://redis.io/docs/reference/sentinel-clients/ to determine valid master "dsn". There is no http protocol here. This code working in production for 3 years for now without major issues

@qlereboursBS
Copy link

Ok thanks, I understand.
However, I have an error when running this code inside the Laravel Reverb library.

I'm intanciating the client as follows:

$sentinelClient = new SentinelClient(
            [
                "<sentinelRemoteIpAddress>:26379",
                "<sentinelRemoteIpAddress>:26380",
                "<sentinelRemoteIpAddress>:26381"
            ],
            "mymaster",
            null,
            $loop
        );
        $masterConnectionPromise = $sentinelClient->masterConnection('/1', ['timeout' => 0.5]);
        $masterConnection = await($masterConnectionPromise, $loop);
        return $masterConnection;

but the connection with the sentinel throws an error:

TypeError {#2995
  #message: "strlen(): Argument #1 ($string) must be of type string, Closure given"
  #code: 0
  #file: "/var/www/html/vendor/clue/redis-protocol/src/Clue/Redis/Protocol/Serializer/RecursiveSerializer.php"
  #line: 27
  trace: {
    /var/www/html/vendor/clue/redis-protocol/src/Clue/Redis/Protocol/Serializer/RecursiveSerializer.php:27 { …}
    /var/www/html/vendor/clue/redis-react/src/StreamingClient.php:101 { …}
    /var/www/html/vendor/clue/redis-react/src/LazyClient.php:127 { …}
    /var/www/html/vendor/react/promise/src/Internal/FulfilledPromise.php:47 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:173 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:221 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:286 { …}
    /var/www/html/vendor/react/promise/src/Internal/FulfilledPromise.php:47 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:180 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:221 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:286 { …}
    /var/www/html/vendor/react/promise/src/Internal/FulfilledPromise.php:47 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:180 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:221 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:286 { …}
    /var/www/html/vendor/react/promise-timer/src/functions.php:163 { …}
    /var/www/html/vendor/react/promise/src/Internal/FulfilledPromise.php:47 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:173 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:221 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:286 { …}
    /var/www/html/vendor/react/promise/src/Deferred.php:45 { …}
    /var/www/html/vendor/react/promise/src/Internal/FulfilledPromise.php:47 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:173 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:221 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:286 { …}
    /var/www/html/vendor/react/promise/src/Internal/FulfilledPromise.php:47 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:180 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:221 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:286 { …}
    /var/www/html/vendor/react/socket/src/TimeoutConnector.php:35 { …}
    /var/www/html/vendor/react/promise/src/Internal/FulfilledPromise.php:47 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:173 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:221 { …}
    /var/www/html/vendor/react/promise/src/Promise.php:286 { …}
    /var/www/html/vendor/react/socket/src/TcpConnector.php:145 { …}
    /var/www/html/vendor/react/event-loop/src/StreamSelectLoop.php:254 { …}
    /var/www/html/vendor/react/event-loop/src/StreamSelectLoop.php:213 { …}
    /var/www/html/vendor/react/event-loop/src/Loop.php:250 { …}
    /var/www/html/vendor/react/async/src/SimpleFiber.php:61 { …}
    React\Async\SimpleFiber::React\Async\{closure}() {}
    /var/www/html/vendor/react/async/src/SimpleFiber.php:71 { …}
    /var/www/html/vendor/react/async/src/functions.php:367 { …}
    /var/www/html/vendor/laravel/reverb/src/Servers/Reverb/Publishing/RedisClientFactory.php:31 { …}
    /var/www/html/vendor/laravel/reverb/src/Servers/Reverb/Publishing/RedisPubSubProvider.php:45 { …}
    /var/www/html/vendor/laravel/reverb/src/Servers/Reverb/Console/Commands/StartServer.php:79 { …}
    /var/www/html/vendor/laravel/reverb/src/Servers/Reverb/Console/Commands/StartServer.php:63 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:36 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php:41 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:93 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:35 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php:662 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php:211 { …}
    /var/www/html/vendor/symfony/console/Command/Command.php:326 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php:180 { …}
    /var/www/html/vendor/symfony/console/Application.php:1096 { …}
    /var/www/html/vendor/symfony/console/Application.php:324 { …}
    /var/www/html/vendor/symfony/console/Application.php:175 { …}
    /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:201 { …}
    /var/www/html/artisan:35 {
      › 
      › $status = $kernel->handle(
      ›     $input = new Symfony\Component\Console\Input\ArgvInput,
    }
  }
}

It's thrown by this method of the RecursiveSerializer:

    public function getRequestMessage($command, array $args = array())
    {
        dump($args);
        $data = '*' . (count($args) + 1) . "\r\n$" . strlen($command) . "\r\n" . $command . "\r\n";
        foreach ($args as $arg) {
            $data .= '$' . strlen($arg) . "\r\n" . $arg . "\r\n";
        }
        return $data;
    }

because $args is looks like this, when it should be a string:

array:1 [
  0 => Closure(StreamingClient $client)^ {#2863
    class: "Clue\React\Redis\SentinelClient"
    this: Clue\React\Redis\SentinelClient {#2822 …}
  }
] 

Do you have an idea why?

Thanks

@qlereboursBS
Copy link

More specifically, I don't understand why the $chain is initialized with $chain = reject(new \RuntimeException('Initial reject promise'));.
I'm not used to use php and promise, can you explain what it does please?

@sartor
Copy link
Author

sartor commented Mar 25, 2024

Here is my RedisPool class for handling connections:

<?php

declare(strict_types=1);

namespace App\Components;

use Clue\React\Redis\Io\StreamingClient;
use Clue\React\Redis\SentinelClient;
use function React\Async\await;

class RedisPool
{
    private const REDIS_RETRY_INTERVAL = 0.5;

    private int $attempts = 0;
    private float $lastTime = 0;

    private ?StreamingClient $client = null;

    public function connection()
    {
        if ($this->client !== null) {
            return $this->client;
        }

        $currentTime = microtime(true);

        if ($currentTime - $this->lastTime < self::REDIS_RETRY_INTERVAL) {
            return null;
        }

        $this->client = $this->tryConnect();

        if ($this->client === null) {
            $this->lastTime = $currentTime;
            $this->attempts++;

            echo date('Y-m-d H:i:s ') . "Redis connection attempt: $this->attempts\n";
        }

        return $this->client;
    }

    private function tryConnect()
    {
        try {
            $sentinels = array_map('trim', explode(',', $_ENV['REDIS_HOSTS'] ?? '127.0.0.1:26379'));
            $sentinelClient = new SentinelClient($sentinels, $_ENV['REDIS_MASTER'] ?? 'mymaster');

            /** @var StreamingClient $client */
            $client = await($sentinelClient->masterConnection('/' . $_ENV['REDIS_DB']));
        } catch (\Throwable $e) {
            echo "Unable to connect to redis\n{$e->getMessage()}\n";
            return null;
        }

        $client->removeAllListeners('close');

        $client->on('close', function () {
            echo 'Redis closed' . PHP_EOL;
            $this->client = null;
        });

        $client->on('error', function (\Throwable $e) {
            echo 'Redis error: ' . $e->getMessage() . PHP_EOL;
            $this->client = null;
        });

        echo date('Y-m-d H:i:s ') . "Master connection established\n";

        return $client;
    }
}

Is use it simply:

$redisPool = new RedisPool();
$redis = $redisPool->connection();
if ($redis === null) {
    return (new Response(504, [], 'no redis connection'));
}
await($redis->rpush($listName, $serializedData));

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants